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ABSTRACT 

Three scholarly efforts, in addition to kindergarten 
through grade 12 research, are necessary" before research and 
development is likely to improve teacher education. First, systematic 
development efforts must be undertaken to' reexamine and translate 
elementary- secondary research findings into articulate sets of 
curricula for teachers. Second, research must be conducted to 
indicated how adults, including, but not limited to, teachers, can be 
taught the skills 'found beneficial to their professional activities. ^ 
Third, Systematic attention must be given to thie ^ay in which goals 
for teacher 'education are determined. The role that each of these 
three re'sesfrch and development areas must play in improving teacher 
education programs is examined, and the means by which educational 
research may be put into practice are discussed,^ (JD) 
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Abstract 



^^.ajLj.^gh considerable progress has peen made in research dn 
teaching, little advancement has been made in using the results of 
that research to improve programs o^ teacher education. The transla- 
tion of research results into practice can best be expedited by sup- 
plementing research on teaching with research and development in three 
other areas: . (1) development work for systematically adapting the 
knowledge accunjulated through K-12 research efforts; (2) research for 
producing widely applicable knowledge regarding efficacious means of 
educating professional personnel, particularly teachers ; and (3) re- 
search and development work for producing 'tenable specifications of 
goals -for K-12 teaching and teacher education. 

The authors discuss the role , that each of these three research 
and developraeAt areas must play in- improving teacher education programs. 
The strengths and weaknesses of two different approaches to re- 
search in professional education are cited. The two approaches in- 
volve (1) descriptive, correlational, and 'experimental studies follow- 
ing the niodel of research on teaching, and (2) Evaluation studies to 
examine the joint effects of programs which attempt to incorporate 
many isolated findings. 

Parti'-ular attention is given to research and development, for 
producing tenable specifications of goals. The recognized importance 
of this area is traced in the research literature, of the past 30 years. 
Three approaches to this work are .examined : (1) using those teaclier 
behaviors which lead to gains on student achievement measures., (2) 
-accepting the judgments of some group or groups, and (3) determining 
which outcomes might contribute to the alleviation of specified -social 
needs. The advantages and disadvantages of each approach are discussed, 
with special attention given : i tRe judgment approach. 
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Preface 

Most, of this paper should be.^o:f interest to everyone concerned. . 
with the 'improvement of teaching; some sections, however, may be more 
teQhnical thaiM.ae general reader would prefer. We er^courage^he 
reader to move qurckty through any section that appears too technical, 

V ^ . - . • 

looking ^for the major points of the argument without attending to all 
the details. "In particular, we have given detailed attention to , 
juagment studies (pp. 28-39) , since ^^h^y are relatively new to they. 
. field of teacher education and readers may desire a more thorough 
understanding this research.^ A crfreTful reading of this .sectiofi is 
not necessary, however, for the reader to ^acquire a sense of our main 
assertions. ' . - * . . • < . • ' - " 



Research and Development Needs * , ^ ^ 

for the Advancement' of Teaph^r Education 

• Judith E. Lanier and Rober.t E. Floden* 

■■ ' . ■■ • * ' . . 

' 'i ' . ■ ' • 

• ■ • . INTRq]5uCTI0N ^ ' > . 

More than' 130 y§ars have passed sinc^ the f^st professional 
school for te.achers .was established. In this/buritry, but prominent ^ 
.educators still report that little is known about how such .a profes- , 
sional school should' be constituted. Turner (1975) surjnised t.hat, ; . 
"in sp^te of recent improvements in research in the fi^ld, ;the amount 
of dependable information available compared to the amount needed to 
formulate more effective policies and .practices in teacher . education . 

■ ' ' * \ 

is miniscuTe"(p.., 107). Many' volumes. of fadings in research on 
.teaching have been published in recent years (e.g., Dunkin & Biddle, 
1974 Good, "Biddle, & flrophy, 1975 ; Travers, 1973 ; Medley, 1977) , but 

• * 

the implications- of these findings for the improvement of teacher 
,e1fucation are'unclear. • As a Vesult, government officials as well as 
educational practitioners have b^come^ frustrated because the dollars 
spent for research in" education have not produced rapixi or sizable 
improvements in teacher education. ^ • \ . / . 

*judith E. Lanier is co-director of- the Institute for Research on 

■"Teaching, director of the MSU School -of Teacher Education, and a profes 
of curriculum and instruction. Robert, E. Floden is -a coordinator of 
IRT'^ Tpacher Education Research Program and an assistant professor of 
teacher education. , ' • . * . . 

,u ■ ■ ■ ■ 



The apparent failure of research to. improve * practice has' prp<Tip ted 
a variety of responses, ranging from recommendations for reduced* fund- 
ing of research on teaching at one end, to requests for increased re- 
search funding, at the other. The position that support should be. 

decreased is taken by some members of Congress, who insist that if- 

* ' ^* " „ 

research is to be funded, it ought to improve practice- AS a 

professional staff member for the Senate Labor, 'Health, Education; and 
Welfare Appropriations Subcommittee said last yea3J, 'We want NIE to 
show us that we are getting the bang. for the" bucks we are ^pending on 
educational research"' (Kerlinj^er, 1977, p. 8).^ O n the ^ other hand, 
researchers in teaching *argue for increased funding, blaming the 
general lacWof research-based change on the limited amount of research 
reports. MeHley, for^ example, in his review monograph* on Teacher Com- 
petence and Teacher Effectiveness (1977) attributes the^Vweak • • * 
connection . . l between research in teacher effectiveness and the 
teacher education curriculum" to "the quality^ and quantity of research 
findings to date" and to the "access to these findings" -(p- 1). 

There is no. doubt that both positions have^ some validity.; research 
on teaching' should - result in the improvement of teaching pradtices, and 
improved amounts and availability of research findings on teact^ing ^ 
effectiveness should result in modification of teacher education 
curricula, fedth positions, however, mistakenly place the burden for 
improving' teaching practice, on findings from studies of teaching in 
K-12 classrooms, where ^ the largest and most visible research on 
teaching has occurred to date, - ' ' 



' ThQ. authors df-^i-s paper posit that such ppsitioiis are dangerous 
simplistic and incomplfeta^, leading to unrealistic expectations and ^ 
inevitable disappointment ... Research on teaching effectiveness in 
K-12 classrooms, alone ^ can not provide the^ guidance needed fojr the 
improvement of teaching practice. If^^teaching practice As " to be 
improved, K-12 teaching research' must be supplemented^y work in at' - 

^least three additional research and development areas: 

1;. Development work is needed for systematically adapting , 

.•^the l^owledge accumulated through'K-12 research efforts, * , 
builditlg isoljaded findings into the larger constellation 
of expertise tuacbers need. Such development work should ^ 
include intensive study ,of the short- and JoBg'"tei'P con- 
. sequences tjiat the added or, delated knowledge units have 

^ on teachets .and learners \x\, their natural environment".^ j ^ 

2 J Research is needed for producing widely .applicable know-. 
• • ledge , regarding efficacious means of educating; professional 
personnel, particularly teachers. ^ • • > 

" ' '3. Research and development work is needed for producing ^tenable 
* specifications of goals for K-12 teaching and .teach€\r educa- 
tion. - Such-goals necessarily represent judgmental (descrip- 
tions of what constitutes "success in teaching." It is very 
4 likely that different operatijonal definitions of teachers' 
V \ . ''work success" have powerful effects on the nature and re- 
,^ suits of inquiry about teachings 

* r • 

The activities -described' above must not bfi^viewed as substitutes^ 
for K-12 teaching research, but rather ''as /necessary concomitant in - 
quiries that have significance in their own right, as well as^ ai? 
essential role in tt>e improvement of teachings, -practice: ' In this; paper 
we will describe the potential each of these research and devG4^pment . 
'areas has for imptovements in -C<^a^lbiiig. Strengths and weaknesses? of 
alternative approaches ineach*area will also be* noted. 

' o \ ^' ' . ' . . / ^ ^ ' 

. . ^ ADAPTING AND TESTING THE EXTERNAL VALIDITY OF^ " 

FINDINGS FRO^l K-12 RESEARCH 

* Historically^ teacher^ educators have drawn on conventional wisdom 



personal experience, common sense, and ^luthority to determine what 

- r • . • , . ' . ^ 

knowledge and actions were needed for enhancing effectiveness in 

teactfingr There was^^ simply no* al terriafive i souijd and useful empirical. 

data were unavailable, and teacher educators t\ad .to rely on their pwh 

judgment. Recognizing' the iffedequacy of judgment alone they' naturally 

lacked self-assuranc^e. Thus^^^when the f Lrst^ data from K-12 teaching 

research , appeari^d , the findings were accepted and !appl led with undue 

Eagerness and haste. Unfortunately , this situatign st ill .exT&ts . , 

O . k , • . ^ • • , \r r 

Findings from K-12 teaching research continue to be accorded greater/ 
i;nportance to teacher "education th^n is warrahted. - 

Present results of K-12 "teaching studies are* still raeaget and' , 
unstable,, although more reliable- and valid -findings are accumulatiag . - 
No matter how much ..the methods and^ results irajirove, howevef , these 

f m ^ . . , ' -I ■ I -J" 

results will not' provide knowledge that is ditectly applicable to the 
durricular judgments required'bf teacher edxicators. Whatever cause' 
and effect linkages might be established between teacliihg and learning, 
the findings- will generally >be limited to maxi'mizing achievement- o-f ^ 

■ ' • . " . f • r 

S;ingle goal.' Teacher educators^, however, must help • teachers cope with 
competing goals and, in this sense,** must be concerned with teaching^ 
teabhers ,hoW to optimize .rather than maximize.,. Unfortunately, tn^ 
'existence of this state of affdirsjc^is better understoad th!an is the 
means to 'cope with-it. ^ 
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'By necessitfy,^ any single set of xesults from research on teaching 
in K-12 classrooms has limi.ted generalizability ; The research process 
forces a sampling of the eoraplexi tie§ of teaching while . requiring the 
researcher to narrowlyj/focus his or her ^view. .The consequence of • . 



narrowing "focus is that the researcher must 'either exclude l-arge seg-/\' 
ments of portent ially important thoughts, actions , and ^aspects iof the 
1. teaching, environment i or place ..severe limitations on saoiple size, so 
that ethnograpTiic or' case, study approaches' can be employed. ^ In'- either 
case, confidence in gener^lf.zing? findings must be]<limi^ed. 

When findings emerge, the'n, it is critical "^that they be applied . 

y ■ ' ' » ■ 

to, and incoifporated into,* ojhe^ settings with existing'^? teaching -com-- 

plexities so that the effects on other' teachers , other .students , and 

other televant variables can be examined. Those v^ri,ables that were' 

. . ' ^ ^" / ' ■ r ' 

necessarily extruded or ignored in . the initial inquiry mus t now be*, \ 



studied; otherwise, this "obviously good'* set of research findings is 

apt' to generate a false s.ensei of coufidengje. - ' " * 

Take,^for e^xample , Rowe' s^ ^(197^^3 > b) ^results from her "wait-time** 

research, -^lowe found that students provided qualitative^Ly better^. 

' . . •* - ' 

responses to. teachers* questions if given mor^ time to think before 

having to- respond . Such- results' appeared to have immediate' relevance 

■for, and transfer to, th^ curriculum of teacher .edjacation/ (After all, 

) ' * ^. •• . . 

teachers ob^ously want their pupils to provide the best possible 
answers.) Teachers were thus taught to'^increase the time they provided 

" . ■ y ■ ■ .• ■ . ' K 

jfor pupil responses. Biib the waiting behar ipr; turned, out to b^ v^r^y 
unstable and short-lived, dropping back when the training and r^nforce 

m^nt periods ended. - .' ' 

\ ' ' - . • '* » . 

While* we can only 3pecula:te, about reasons for the lack of 

more^pefmanent change in the teachers ! waiting ^behavior , it seems 

reasonable to assume 'that sonie competing goal, 'such as that "of ^main- 
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taining group attention, was more important, " The fact that this 
possibility was overlooked in the application of Rowe's findings is 
illuminating; it illuntrates the need fqr further examination of the 
appropriateness of applying K-12 research findings to teacher education 
curricula. 

The response to Flanders' work (1970) provides another example 
of direct and perhaps premature modification of teacher education 

curricula on the basis of K-12 research on teaching. Many curricular 
modifications were made in the content of teacher education to incor- 
porate instruction about Flanders' interaction analysis system. In 
some cases, for example, instruction on classroom management was 
discarded and replaced with" exercises for learning proper scoring 
techniques for the Flanders' matrices^, Recent time-on-task findings 
(e.g., Harnischfeger & Wiley, 1976) might raise questions about the 
appropriateness of this curriculum shift, but these questions aside, 
it is instructive to consider the pssumptions behind this decision to 
change curricula. '"^ ■ 

The first assumption was that teachers would be more effective if 
they were indirect rather than direct, a conclusion that highly over- 
generalized thai research findings, given the contextual limitations of 
the empirical Work. Tt was also assumed that adapting and using the 
research methodology, i.e., the Flanders interaction analysis system, 
would make teachers more indirect. 'Finally, it was assumed that 



^various studies of teaching, in 'fact, (Kounin, 1970; Brophy & 

Evertson ^976 have Indicated that pupil boredom and lack of .nvolve- 
mlnt leads to distraction and disruptive behavior. .Students who wait 
Toller for each other's responses may well become tired with waiting. 
Thus whiL Che teach^'r is helping one studer • to think and produce 
ITtter responses, he/she may be inadvertentl, contributing to the 
disruptive behavior of others. 
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•whatever was removed from the teacher education curricula to make room 
for Instruction in Che Flandersi,,,s>;s te;n was less significant to effective- 
ness in teaching and learaing. None of these assumptions was necessarily 
correct- Systematic development and evaluation would have given decision 
maTcers better data and more confidence for exercising their judgment 
on wUat to incluw*e or exclude in the curriculum. 

In addition to validating findings, developmental work would also 
lead to improved dissemination, i.e., the translation of the findings 
into more useful and powerful forms. It Is common knowledge that 
verbal dissemination of research findings through written reports.> or 
oral presentations at professional meetings is not terribly effective. 
It is alsp commonly knovm that "teaching by telling'* and ^'learning by 
listening*'' are typicaJLly insuf if Lcient means of improving professional 
practice. .Thus, systematic design of instructional strategies, materials, 
and field trials would help narrow the now classic gap between research 
and practice. 

Feiman (Note 1) emphasized the practical limitations of direct 
use of K-12 teaching research findings in a corriment made during an 
IKT presentation in 1977: "So Kounln finds that 'withltness' is impor- 
tant: for keeping children on task; there's certainly more to the next 
step than telling teachers that it is Important to get with it." 

The results of K-12 research on teaching do^ not directly translate 
into effective forms of teaching practice. Researchers, practicing 
teachers, and teacher educators must work together in the mobilization 
of a well-planned development effort. Such an effort would: (1) 
accumulate the various findings of K-12 research on teaching; (2) study 



8 

the interactions among the variables described in the sets of findings; 
(3) investigate the degree to which the findings hold in a variety of 
teaching and learning environments; and (4) incorporate the findings 
into the larger constellation of knowledge and skills needed by 
teachers, so that dissemination will be effective • Development work 
of this kind should facilitate the translation of research into prac- 
tice. 

RESEARCH ON THE EDUCATION OF PROFESSIONALS 
Admittedly, research on K-12 teaching will assist in the acquisi - 
tion of knowledge about effectiveness in teaching children and youths • 
Emphasis is placed on "assist" to stress the inadequacy of K-12 re- 
search contributions, alone, and toimply that because of inherent 
weaknesses in this body of research, additional modes of inquiry are 
needed. We emphasize "acquisition of knowledge" to make clear that 
the availability of knowledge does not alone insure its effect on 
practice. 

We have discussed ways in which development can address the 
translation and incorporation of K-12 research findings into teacher 
education. If these steps are taken, the prospects of converting 
qualitatively better research results into practice should be improved. 
Again, however, we must stress that the set of necessary inquiries, by 
itself, will not be enough to adapt sound knowledge into practice. 

One area in need of inq^isry is research on the teaching of teach- 
ers ♦ Knowledge available to the research-on-teaching community should 
also be available to the K-12 teachers, but we know little about 
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transmittiiii^ the research knowledge in ways that allow teachers to 
successfully modify their practices. Knowing that the ability to 
read is a desirable skill to give children does not Indicate he 
children should be taught to read; similarly, kno^/in>:. that teachers 
should be taught to provide direct instruction does not indicate how 
teachers should be taught to provide direct ins truct it)n. Without 
research results on the methods for teaching K~12 teachers, the re- 
sults of research on teaching 'cannot be used to improve K-12 teaching, 
Brophy '(Note 2) states the problem succinctly: "Though findings from 
the research.on K-1 2 ^ teaching effects have important relevance for , 
te<3cher educa tirOtC^Tt is unrealistic to seek or expect direct or 
universal statements about good teaching to emerge from this research," 

Some might argue that K-12 research is^ sufficient to indicate how 
teacher educators should instruct, and that the research results do 
have general applicability and can guide the selection of methods 
^ of teaching teachers (e,g., some might believe that research showing 
the value of direct instruction for youngsters is directly relevant 
for teaching adults). While admitting that K-12 research does not yet 
provide adequate guidance for teacher education, these challengers might 
attribute this state of affairs to the size of the research base, imply- 
ing that what is needed is not a different type of research, but rather 

more K-12 research . / 
. 

These arguments can be refuted on two different grounds, the first 
focusing on correctable weaknesses in current K-12 research and the 
other centering on weaknesses inherent in K-12 research. .The current ^ 
correctable weaknesses are the limited generalizability across socio- 
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economic groups" and the correlational nature ol: the results. The 

inherent, weaknesses are the distinctly different subject populations 

rand subject matters oF the research, and the Influence of teacher. 

trai.ning methods on. the associations found in K-12 researcih. ^ 

Current research on teaching K-12 is substantially limited to - 

a small segment of the K-12 population* - 

For reasons mainly connected with the fundings strategy of 
the U.Sr Office of Education, most of the research • . . 
was done in one segment of the school population in 
classes of Grade III or below in which most of the pupils 
come from homes of low socioeconomic status. To what ex- 
tent these findings apply to pupils with other backgrounds 
or in other grades is not known. - What evidence we have about 
pupils of high socioeconomic status and pupils in the higher 
grades indicates that results " from one group do not always 
apply to another. (Medley, 1977, p. 2) 

Since many teachers are drawn from a higher socioeconomic class than 
the students studied, results frbm current K-12 research on teaching 
might not^providk guidance for teaching 1*eachers. However, this lack 
of generalizability across socioeconomic classes could be alle^^iated 
wTtWin the existing framework of K-12 research by conducting, more re- 
search with children from higher socioeconomic groups. 

The correlational rather than experimental nature of current re- 
search also limits applicability to teacher education.' Following the 
familiar dictum that "correlation is not causation, 'V it cannot be 
known whether the assocl ations found in the correlational research 
represent causal relationships or merely some form of spurious correla- 
tion. If^they are spurious, there is no assurance that changing teacher 
-behavior in particular ways will produce associat^ changes in children. 
Although statistical pi-ocedures can "be used to elimitiate some possible^ 
sources of spurious correlation, no procedure'is foolproof. 

ERIC • ■ i:\ 
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Again, however, this weakness Ln current research could b'? 
eliminated within the framework of research by conducting ex-r 

perlmental studies. In fact, investigators have recently begun to ^ 
conduct experimental tests of the associations discovered in K-12 
research (Borg, 1970; Gage, 1976; Gage & Crawford. Note 3). Sufch 
investigations are' frequently linked closely with developmental efforts 
'such as those described in the previous section. 

Although the two difficulties in K-12, research identified above 
can seemingly be alleviated by modifications within the K-12 frame- 
work, dther obstacles to application cannot be^overcome without em- 
ploying another type of research. These obstacles are: (l).the prob- 
lem that, results might not generalize to an older population iWning 
a different subject matter, and (2) the problem that all methoas of 
teaching particular behaviors or skills may not necessarily reap 
the previously-associated advantages . 

It is difficult to believe that results obtained with students in 
elementary school will apply equally well (if at all) to adults of 
college age preparing to teach (preservice^ and to older adults al- 
ready teaching (inservice). One of the major lessons derived from ^ 
recent, developmental psychology is that children are not .merely little 
adults. Children have their own ways of thinking and learning; intro- 
spection about adult perceptions cannot inform us about the perceptions 
of children. A corollary here is that adults .are not merely big 
children. Adults have different perceptions an_<? needs than do children, 
and a strategy successful for teaching children will probably not be 
successful for teaching adults. Medley (1977) pointed out that re- 
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suits obtainj»d from young elementary school children probably do not 
apply to students in the higher grades. That assertion is even more 
relevant when the students are adults. No amount of research in K-12 
can determine whether the results of K-12 research will be applicable 
to older groups.- . . 

Differences between subject matter taught in K~12 and in programs 
of teacher educatiorisalso limit the applicability of K-12 findings to 
preservice and inservice teacher education. The standard curriculum 
in elementary and secondary school does not include subject matter 
sucK^as classroom management, educational psychology, or methods of 
teaching mathematics. ' These subject matters bear little resemblance 
to any part of the K-12 curriculum, and no amount of K-12 research can 
discern the ^direction or extent of the differences; hence, a different 
type of research" is necessary to provide information on how teachers 

can bes't be taught. 

Finally, the instructional framework in which teachers acquire 
behavioral modes may alter the .effects their behavior has on the.child 
they teach. Rjeturn, for a- moment, to the example of Rowe's work. The 
effect of wait-time on school children may be substantially different 
if teachers are taught not only the importance of wai!:>i;iiae, but also 
to consider when waiting is most appropriate ^& -jpposed to being be- 
haviorally conditioned to wait , three seconds Jot a respon^se a ques- 
tion^. The" reflective teacher may occasionally realize that his/her 
question wt^s poorl^r phrased and rephrase it immediately. The be- 
haviorally-trained teacher would just Wait three seconds, in which 
time a student raigJit ma^^ • a digressive response. Divergent K-12 re- 
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search results induced by different methods educating teachers cannot 
be investigated within the framework of K-12 research; the investigiw 
tion of the, differing effects requires a move into' research on the 
professional education of teachers* 

Research which would avoid the difficulties inherent in K-12 
research need not be restricted to teacher education. While K-12 
research cannot be generalisrcJ to teacher education because of the 
differences in age and subject matter, supplemental research in areas 
of teaching with similar age groups and subject matters might well be 
applicable to teacher education. Areas of professional education such 
as medifcal education, the education of clinical psycholcjglsts , and 
teacher education iiay be sufficiently similar for results in research 
on teaching in una area. to be applicable to the other areas. In each 
of these areas, adults are taught some combination of discipline know 
ledge and technical skills in a setting which combines forms of didac- 
tic group instruction and supervised field experience . ^ Although the 
areas undoubtedly differ in other respects, these striking similarities 
in age level , subject matter ."""and Instructional settings make a high 
degree of generalizability across areas likely, or at least munh more 
likely than generdlistiblllty from K-12 teaching to teacher education* 

By broadening the scope of research to include all these areas of 
professional education, investigators could profit from many new re- 
search findings and from the variation in' specific instructional tech- 
niques that other fields have developed.. Assuming that the findings in 
research oh other areas of professional education are relevant to and 
perhaps even equally valid for teacher education, progress in teacher i 
education could be expedited. In addition, if instructional techniques 
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frorn the other areas were introduced into teacher education, the benefits 
to teacher -education could be^increased tnanyfold. • 

Why, it might be asked, should .this extension be limited to re- 
search on professional education? Why not carry it further, to encom- 
pass all research on adult learning and on the teaching of adults? 
After all, the age groups are the same in each case. 

.The answer here is obvious. Although the age groups are similar, . 
the subject matters and instructional settings vary widely. Because 
of these substantial differences, results would probably not be • • 

generalizable between professional education and more general adult 
education. Restricting the scope of research to professional education . 
Strikes a balance between similarity in situations and broadness of 
situation. The scope would be narrow enough to avoid a breakdown in 
general izability, yet broad enough to enable researchers to finc3^. 
audiences for their work ^and to find a sufficient variety of meaning- 
ful problems, f 

In conclusion, we bjelieve that the current weak link between K-12 
research* and the practice, of teaching and teacher education would be 
strengthened by research on the education of teachers. It is our 

further contention that this research' might be conducted most expeditious 

\ ' ' ' ' 

ly by including in its scope all types of professional education, 

\ . ■■ . 

Approaches to Research on Professional Education 

Two general approaches may "be taken toward research on professional 

education, each with distinct strengths and weaknesses. The first 

approach follows the general methods of research on teaching K-12 

learners and attempts to investigate the separate effects of ^istinct 



components of teaching through descriptive, correlational, and experpL- 
mental studies. The second approach attempts to assess the aggregate 
effects of a complex combination of components through program evalua- 
tion. . The research approaches complement, each' other, and both can c >n- 
tribute to the improvement of teacher, education. ^ 

The approach taken in .K^12 research has the advantages of an analyti- 
cal approach. Components are considered separately, and the effects 

• ■ • ■ • ' '* 

of an entire progran can often be predicted frqjn knowledge of those 

coroponetlts , provided ':hat interaction effects are relatively^ small • 
This approach has as an added asset, familiarity. Many behavioral 
scientists understand the techniques Involved and, as a result, might 
be able to apply them to the ^rea of professional education. \ 

Research on teaching K-12 has often been severely criticized, 
however, particularly on the grounds thit studies have been almost ex- 
clusively correlational, rather than^ experinient:al The lack of ex» 
periraental studies may be due to the great difficulty in obtaining ex- • 

perlmental control over the assignment of treatmecnts to students. 

' ' . • '\ ' 

Although this control was achieved in tiie Follow Through , Study (Stebbins, 

. ■ * . ' I 

St. Pierre, Proper, Anderson, & Cerva, 1977), it accomplishe]d onlV 

at great expense. Control over, the assignment of subjects may be easier 

^- • - I " 

to obtain in research on professional education, wher^ scheduling Is 

less difficult and the cooperation of the participant'^S and approp^riate 

administrators is more easily obtained. j 



*This criticism may subside as more experimental studies are \con- 
ducted. It is interesting to note 'that the most * prominent experimental 
studies that have been or are being conducted Borg, Kelley, Langer, 
& Gall, i970l Gage, 1976; Gage & Crawford, Note 3 — would be best \ 
classified as research on professional education. The only experimental 
intervention lies in the teaching of professionals. That is, there lis no 
^''■'.rec.t intervention in the classroom; . changes are ta occur only .inditectly 
^J^(^* a result of the teacher education. o o 

"~ ■ \ ■ ■ - 



* If Interactions among components of an educational expeiJ^ence' 
are powerful (as suggested by Cronbach, *1975) , studies of only a 
. single factor have ^limited value in determining the Effects of that 
'factor- in the context of a complex educational program. More prer 
cisely, research which considers only main effects has limited pre- 
dictive power when interaction^ are larg.e. Information about inter- 

actions can be ^obtainaed by investigating the joint effects of cora- 

... fi- . 

poneats in a more elaborately de^iigned > study . However, the number of 

' » . ■ * 

subjects needed for a' study of even a siriall number of factors and their 

interactions quickly becomes prohibitive. ^ " ' - 

Program evaluations- cafn be us<ed -tio assess the combined effects it 

of a complex combination of ^ compor>crJ:s , although the individual cpn- 

Cributions of these .components c^hnoc be determined. Tfie^Fbllow 

Through Study might better be thought of as a program evaluation than 

* • 
as a research study investigating individual factors; the same-might 

be said of the current study by Gage aud Crawford (Note 3). 

Program evaluations also provide inforniation on the components as 
actually implemented, rather than as experimental mani^-^ulations in a 
research setting. Often, components change in aubtle (and not ao subtle) 
ways when they are administered as part of a larger program by people, 
who are administrators rather than researchers. 

The disadvantage of program evaluations is that they yield results 
which often have extremely limited generalizability. The evaluation' 
might indicate hew well an entire program worked in its particular setting, 
but it has lit:le^ pc^wer to predict how parts of the program would work 
when adapted to othe?.- settings. Still, program evaluations are essential 

er|c ; • . 



to the "^Improvement of teacher ^ducatio^, 'since ^every program of 
professional edupation is corapdsed of many components within-an ^ : 
administrative structure; it is unreasonable to believe that* the 
interactions, which are likely to be>^arge, will have been assessed 

. ; 

by previous research • . ^ 

Again,/lt is important to emphasize the promise ^of, ^nd need for, 
.coordinated efforts in this area. If common^ metrics are agreed upon 

' • : . . j. ■■ 

for evaluating various -teacher- education programs , and if data are ' 

\ ' . , - ■ ■ ^ ■ • - : . ' 

•J ' y • . - >, 

accumulated and shared they caji readily be studied w^ith new meta- 

♦ ■'^ . • . • * " 

analysis techniques (see'GlasSj^ in press). New approaches would 
afford- increased potential for generalizing findings from program 
evaluation studies.. .< - ' 
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RESEARCH ON THE DESIRABLE OUTCOMES OF | 
. ■ K-12 TiEACHING AND TEACHER EDUCATION 

Our third and final -suggestioji f or. bi^ilding more effective knaw- 
Tedge bases in teaching — perhaps, the most difficult and challenging ^. 
one ~ is to develop tenable • definitions of "effective teaching." While 
this. task is ^probably the most critical one for improving research "in 
teaching both' K-12 and pro'^^ssional population^, it'is the one most 
neglected by the research and development community. No research on 
K-12 teaching or study of professional teacher education (preservice - 
or inservice) can ever be vsed to construct better programs for teach- 
ing teachers unless some decision is made abqut the definition .of- "better 
and' teacher education programs cannot be compared in quality: until such 
decisions are "made explicit. ^ r ]. ' . 

Partial definitions of what constitutes effective teaching%can be .. 



inferred from the goals*. teacher education programs seek to acGorapllsh. 

. • , . j» ' 

Those goals, in turn,*' should detei^raine the outcomes , measured when . 

V ■ r ■ I ^ 

program effects are studied. Justification of the par ticular choice 
of outcoraes>is seldom made, but outcomes must be chosen before a . ^ 
program is evaluated or research' conducted. We argue ^hat: (1) the 
choice of outcomes implies selection of definitions'* of 

effectiveness, 'and (7.) the choice • should be explicjLt and carefully 

' ■' ' • • , 1 ■ ' ' . ■ * ■ 

justified, i.e., the definitions should ^be tenable . 

Researchers on teaching focus on- teaching outcomes' and charac- 
teristi^- that, in some sense, they consider Important or desirable. 

• ■ * 

The outcopie variables or salient characteristics selected by re** 
*3 " searchers are either de facto operational definitions bf work success 
in teaching, or they are expected to be related to work^ success . 

^ -Operational definitions chosen by the investigator for purposes^ 

' ' ' . ' ' - * *■ . > 

of scientific inquiry may or may no t^ be valued by dthers in., the educa- 

^ . ' . . • ^ ^ . ^ , . 

tional ' community. For example, st^ndar^i'zed^'tests ate often used as 

outcomes measures in, research on K-12 teaching', because they are 

thought to measure* student attainment on generally recognized educational 

•goals*' There is, however, a notable lack of consensus among educators 
on the adequacy of standardized tests as measures of teaching work ^ ' 

1 success. Nevertheless , a number of ^investigators focusing on K-12' 
research continue to rely heavily on these i:ests.. ^ ■ . 

No standardized tests are available to measure desirable out- 
comes of teacher education. The National Teachers Examination does' not 
have a broad Base of support, so researchers and desigtiers in teacher 

• * - o - * 

education .tjrpically look ^elsewhere for guidance. They frequently rely 



on "expert" judgments of supervisors- and a'dininistrators . In S ine cases, 
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researchers iiive^stigat ing the teaching of teachers "avoid the question 
of whaC cohi.stitutes.\§ucces^s'.in teaching and select a "ptoxy" variable 
or set^f variables, instead^, 'Often, these proxy variables .simply 
show relationships among elements in theitraining program (e.g. , ' 
success in one course related to success in an ensuing course/.^ In' 
essence; they substitute success in the program, itself, for work ' 
success' in teaching. In s'uch cases, ^Attention is givern to program 
components, and the choice of teaching success outcomes is left 
either to the judgments of researchers qn K-12 teaching or; to who- 
ever supervises prospective teachers, during the practice teaching ex- 
perience. ' - \ : ^ ^ 

. The lit-tle research on teaching inservice teachers that has been 
conducted h3s al6o tfended%to neglect the development 'of ^nable defi'ni 
tidnS' of' teaching: work success.' There are,^of course, some notable 

V 

exceptions, such as ,the work of. Borg and his colleagues (Borg, Langer, 
& Wilson, 1975; Borg, 1976y. ■ 

The major emphasis in research oh teacher- education, then, has 
been on teacher preparation and on describing and analyzing program 

elements of particular practical interest,- Thus, moou teacher educa- 

■.■ ^ » * *^ 

, • ^ 

^ion research has ignored the problem of choosing tenable outcomes, (, 

concentrating instead on procedural questions such. as: (1) the amount. 

of subject matter or educational foundations knowledge needed for^ 

success in a particul.^r program, (2). the effects of- diverse types and 

amounts of field experience on student and supervisor: Judgment, (3) 

the effects of competency-based teaching methods, (4) modeling and 

feedback procedures, and (5) simulate.d training exercises on student 

Qihievement of coujse objectives • <- ^* 
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It is important to note that the focus tends 'to be on methods 
of teacher preparation. Although research into such methods has 
obvious importance for decisions about instructional programs, it is 
highly debatable whether useful information can be gathered without 
a more tenable specification of program goals* Questions on methods 
of teacher preparation are relevant and, as yet, unanswered, but the 
answers will tell us little about what facilitates effective teaching 
until the goals of teacher education have been more carefully determined • 

Turner, in his "overview of Research in Teacher Education" (1975), 

supported this view. He stated: "The objective of analyzing teacher 

work success is to clarify what one is trying to optimize" (p, 87). 

He also supported a shift in research emphasis from the focus on 

program components to the outcomes. 

Moreover, the classical view of research in teaclier education 
has not shown remarkable results over the 'past 30 years, and 
an alternative to it may be regarded as desirable from the 
prospective of the overall research strategy in the field, 
the optimization of teacher work success, (p. 91) 

' Turner was by no means the first to recognize the urgent need for 

clarity of goals in research on teaching* A generation earlier, 

Rabinowitz and Travers (1953) had stated: 

* Any study of ability depends upon a conception of what 
constitutes successful functioning. Before definitive 
research on the factors associated with the effectiveness 
can be pursued, "It must be possible to specify some cri- 
terion through which effectiveness may be identified* Re- 
search based upon a clearly unacceptable criterion 
cannot -produce rr.sults of ^ny great significance. Why 
then do we not develop more adequate criteria of teacher 
effectiveness? (p* 212) 

Six years before that, Flanagan (1947) noted: 



It is impossible to study the requirements for success in 
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. an activity without defining the c.ctivity, A complete 
definition of what is meant by success in the activity is 
practically identical, with a statement of the procedure 
for obtaining a criterion. 

As recently as 1976, at the meeting of the Invisible College of 
Researchers on Teaching (held at the Institute for Research on Teaching), 
prominent mernbers of the research community agreed that a major reason 
for disappointing research results was the weakness in conceptual clarity 
about outcomes and criterion measures. 

Of central interest here is that the neglect of outcomes in re- 
search on teaching is not due to a lack of awareness of tue problem, 
or even to disagreement (at a general level) about the form of the 
solution. Why, then, has the recommendation been acknowledged so 
strongly in words, yet ignored in deed? 

Thirteen years after Rabinowitz and Travers (19.53) urged researchers 
studying teaching to begin concerning themselves with criterion questions 
and necessary value judgments, Travers reflected on the lack of progress 
toward this end. In his most salient observation, he urged researchers 
♦ to avoid predictably "dead-end" approaches to outcome specification, such 
as the standard behavioral objectives approach and the Bloom-Krathwohl 
type of taxonomic work. Commenting on the weakness of the behavioral 
objectives approach, Travers (1966) stated: 

Since Tyler many years ago wrote on the importance of 
defining educational objectives in behavioral terras, vast 
lists of behavioral outcomes have been prepared. Those 
who have engaged In this activity seem to agree on the 
virtues of undertaking this task and also on the unwieldly 
and unuseable produce which generally emerged from it. (p. 113) 

It should be noted that this warning was issued prior to the emergence 
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of Che competency per forma nce-base d mo vement_j.n teacher education* 
Yet, awareness of the problems and probable outcomes, alone, was in- 
sufficient to prevent, the production of seemingly endless lists of 
disorganized bits of "desirable" outcome behavipr. 

Travers (1966) also condemned taxonomies built around "response 
inferred" characteristics. He criticized the selection and- organiza- 
tion of tasks on the basis of response-inferred characteristics (such 
as knowledge and comprehension) just as strongly as he admonished the 
collectors of behavioral objectives* ' 

They are of little use in the development of theory of in- 
struction. Since the taxonomies classify tasks on a basis 
which lacks any established scientific utility, ttje applica- 
tion of the taxonomies results in a bulky and unraatvageable 
collection of tasks which generally cannot be used for 
scientific purpose. The product which results is reminiscent 
of those bags of unsorted foreign postage stamps which 
philatelists like to rummage through in the hope of finding 
some rare item. (p. 114) 

To avoid the behavioral objective problem, Travers recommended 
the construction of a taxonomy of tasks with a scientific base and 
scientific utility for building theories of teaching. He suggested 
the use of a "system of scales" a set of high-order instructional 
tasks selected on the basis of a value judgment of their importance to 
learning specific content. He urged researchers to give special atten- 
tion to subject matter specificity, as opposed to taking the generic 
approach of many competency-based efforts. By this suggested approach, 
all lower-order tasks would be subsumed beneath the higher-order ones; 
the scaling would define the outcomes, and the resulting ordered systems 
of tasks would, in turn, define^ the outcomes of instruction^ 
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Travers (1966) described another way of viewing the process: 

All outcomes of ^education can be viewed as the consequences 
of the production of rule-regulated behavior. Outcome^s can 
be specified in terms of the rules that regulate behavior. 
These rules can represent narrow rules which specify the 
orderliness of behavior with respect to single very narrow 
tasks, in which case the outcomes of any educational program 
have to be specified in terms of a large number of rules. 
On the other hand, the rules may be broad in scope and refer 
to general lawfulnesses covering a wide range of tasks 
in which case. the objectives can be specified in terms of a 
set of relatively few rules. While procedures for defining 
objectives in the past have implied that many rules have to 
be specified, the proposal here is th^at these be organized 
into broader rules covering a wider range of events, (p. 115; 

A review of the professioral activity that followed Travers' 

recommendation reveals no attempts to act upon his advice; instead, 

researchers have engaged in a cycle of activities: (1) reasserting 

that clarity of goals is impor.^ant, (2) getting others to agree, (3) 

making lists of p-mportant outcomes, (4) reviewing the results and 

deciding that the effort was a waste of time, and (5) concluding that 

the original idea to clarify goals was a bad one. 

The conclusion, however, should not be that the concept of 

clarifying outcomes is inappropriate, but rather that the attempts at 

implementation have been faulty. We agree with the scholars of each 

decade who have urged pursuit of more appropriate definitions of work 

success in teaching. Further, to avoid repitition of the mistakes' of 

the past, we call for an indepth analysis of several alternative 

approaches to goal specialization, with the hope that they vill provoke 

both criticism and creative suggestions. The problem is not an easy 

one, but we are optimistic; the task we face, although complex, does 

not appear impossible, and we are hopeful that new technologies may help 
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us find the clarity we n^ed. 

Approaches to the Choice of Desirable Outcomes . 
Three general methods of choosing desirable outcomes may be 
identified: (1) using those teacher behaviors which lead to gains 
on achievement measures, (2) accepting the judgments of some group 
or groups, and (3) determining which outcomes might lead to the 
fulfillment of specified social nerds. Each of these approaches — 
along with its advantages and disadvantages will be discussed. 

Behaviors Leaditig to Student Achievement Gains ^ 

Outcomes for teacher education^ are most commonly, selected from" 
those teacher actiona and behaviors which have been shown to produce 
gains on pencil and paper achievement measures, particularly on 
standardized achievement tests in reading and mathematics. Examples 
of this approach can be found in almost all articles reviewing re- 
search on teaching in K-12 (Medley, 1977; Rosenshine & Furst, 1973; 
Rosenshinei - 1971, 1976) and in many texts on educational psychology 
(Gage & Berliner ^ 1976; Good & Brophy^ 1977). Not all researchers, of 
course, consider standardized tests the best indices of desirable out- 
comes.. Kounin (1970) and Borg (1976) , for example, are more concerned 
with teaching actions which decrease student disruptive behavior; such 
studies are relatively rare, however. To simplify this exposition, we 
will concentrate only on standardized tests of reading and mathematics 
as guides for choosing outcomies. 

The assumption behind this method of selection is that gains on 

/ 



ERJC 3^ 



25 



reading and mathematics' tests are desirable educational ou':comes . 
While these gains are not necessar^ily the only desirable outcomes, ^ 
they are the most easily measured and have become the primary ones 
toward which teaching actions have been geared. • Proponents of this 
approach admit that the behaviors associated with these gains will 
not produce all the desirable educational outcomes; but they argue , 
that at least some desirable outcomes will be produced and challenge 
others to propose a method which can do more. This approach may not 
lead to the best possible" teacher education program, but it seems 
likely to produce improvement and allows for immediate action. Re*- 
suits that h^ye been obtained in K-12 research can be used immediately 
to establish outcome measures for research on profess J.onal education. 

This approach to choosing outcomes, however, can be challenged 
on at least three different points* First, are the tests used reasonable 
measures of the basic skills that have values generally agreed upon? 
Second, does a generally-yalued set of basic skills even exist? Third, 
is a general agreement that the basic skills are valuable a sufficient 
reason to believe that they^ are, indeed, valuable? 

Standardized tests have received substantial criticism in recent 

years on .the ground that they are poor measures of the valued basic 

skills. ' ^ 

The standardized achievement tests are also highly correlated 
with standardized intelligence tests, thus causing us to 
wonder e:cactly what kinds of items are really used in these 
tests. Furthermore, the tests are usually group adminis- 
. tered multiple-choice tests. When working with young, 
bilingual, or lower socioeconomic status children, there is 
a serious question about whether many of the children are 
being appropriately tested. (Berliner, 1975, p# 4) 

In The Myth of Measureability (Houts, 1977), a collection of articles 
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taken from the Principal > several ^uthors suggest that 
close examination of standardized tests reveals the tests bear little 
relation to skills and concepts valued by the public. While these 
claims might be somewhat exaggerated , they doTindicate that standardized 
tests should not be blindly accepted as appropriate measures of 
education. , ?t. 

A related, but' even deeper criticism of this approach to choosing 
outcomes is that there is little agreement about which particular set 
of skills should be considered basic. In mathematics, tor example, 
considerable attention was given during the 1960s to the question of 
vhether.or not material such as elementary set theory wa's part of the 
mathematical basic skills. The question of the compos it; ion of the 
basic mathematical skills was never answered, however. As recently as 
October, 1975, a National Institute of Education Conference directed 
itself to the question, 'Vhat are bas ic mathematic'al skills and learning? 

(NIE, 1975). ^ 

It is proposed here that the determination of what 
mathematics is most worth learning is a task that will 
\ require careful and systematic study from the perspectives 

\^ of several interes^t groups. (H^lms & Graeber, 1975, p. 70) 

^ The ch'*' ..enge to describe basic skills and learning in school 
\ mathe:-uitics is an assignment full of pitfalls. In the past 
five years, hundreds of mathematics educators, school systems, 
professional groups and the National Assessment have been busily 
cot^posing taxonomies of fundamental objectives for mathematics 
iri'fe'truction at various grade levels. With few exceptions, these 
effoVts to establish a reasonable list of basic ^skills hav,^ been - 
failures. There has been no general agreement among the compet - 
ing groups . >foreover, the* implementation of the various lists as 
curr.iculum guidelines threatens to produce fragmented mathematics, 
programs that resemble occupational training more than they 
resemble education in mathematical methods and understandings 
likely to be of long range value, (Fey, 1975, p. 51, emphasis added) 
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These mathematics educators may have exaggerated the di:^erences o£ 
opinion concerning composition of the basic skills; however, their 
commencs do suggest that no particular 3et of skills has general 
sanction. 

A further weakness with the use of standardized tests ,.s guides 
for choosing o^t^omes ii3 that the different tests do not even measure 
\ the same skills* Recent analyses indicate that the most widely used 
tests cover significantly different content (Porter , Schmidt, Floden, 
& Freeman, 1978). Hence, even if some single ^set of basic skills could 
-be identified, not all the major achievement tests would measure progress 
on that set of skills- . . " 

Finally, it must be recognized that general popular sanction is 
not necessarily an adequate reason for accepting an outcome as valuable. 
The fact that a l.arge number of people might esteem an outcome does 
not automatically confer value upon that outcome. Here, "valuable" 
is taken to mean '/beneficial to members of society as specifie.d by. 
explicitly stated criteria," rather than "capable 'of being valued." 

The distinctioui between "valued" and "valuable," or "desired" and 

"desirable," or "wanted" and "worthwhile" becomes clearer if one con- 

I 

siders that the majority of people may often be led to value particular 
outcomes by adver tisi-Rg and publicity. Such fabricated values are un- 
stable and probably do not reflect the real beliefs of the 'people . 

Popular opinion is not the determinant of -^aluabili ty, yet the 
argument for using tests -because they are accepted is based on the 
assumption that popular opinion does lead to value. Critics of the 
accepted-test approach question the assumed connection between valued 
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and valuable and insist that some alternative reason must be found far 



choosing the outcomes and goals of teacher education. 



In suThmary , identifying goals for teache r educ ati on by associa- 



tionSjjith student gains bpi standardized test 
familiarity, convenience, and- general accept 



;3 has the advantages of 



ability. ' But this 

approach can be attacked on the ground that jthe outcomes have not been 
adequately , defended as valuable, i.e., the cutcoraes are not tenable. 
A similar trade-off will be .seen in the discussion of the next general 
approach for determining th^aptpjcopiriate outcomes, of teacher education. 



Judgment Studies 



A second approach to determining appropr^Late outcomes of teacher 

\ y 

education is to idetjtify and accept the best .j|udgraents of some group 



or groups of individuals deemed worthy of choosing outcomes. Such 
groups might include parents, teachers, school officials, teacher 
educators, behavioral scientists, or some other constituency. The 
judgment of a particular group might be obtained by direct questioning 
or indirectly through a questionnaire or behavioral study. 

This approach has two obvious strengths. First, it automatically 
generates political support for the program of research and development 
which follows. If the goals for teacher education are solicited from 
a certaip group, that group would seera bound to support research and , 
programs striving toward those goals. Second, the judgment-studies 
method resolves one of the major shortcomings of the standardized test 
approach. By empirically determining what goals are valued, the 
question, "Do people really value the goal?" is answered in the 
affirmative (at least as -far as the empirical procedures are successful). 
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Furtherjiiore, since administrators' and Leacher educators must make \ 

judgments about goals in their current work, anyway, those day-to-day \ \ 

.judgments should at least be made explicit. By casting these judg- > 

ments into a ^^mework.of empirical study, greater clarity can be 

obtained aboyt who the judges are and what basis they are using for 

judgment - , « ^ 

• For purposes of teaching teachers or studying the teaching of . 

teachers, ah explicit conception of teaching that presents the ac- 

company ing "ought" views appears critical. Rabinowitz and Travers (1953) 

recognized this 25 years ago and cited many experts before them who 

shared their view: ^ ^- 

There is no way to discover t3*he characteristics which 
distinguish effective and ineffective teachers unless one 
,has made or is prepared to make a value judgment. The 
effective teacher does not exist pure and setene, avail- 
able for scientific scrutiny, but is instead a fiction in 
the minds of men. No teacher is more' effective than ^ 
another except as someone so decides and designates. 
Teachers are real enough, and methods are available or can 
be improvised to study thes^e real teachers. But the effec- ^ 
tive teacher is only an abstraction. 'The process of designating 
any particular teaching practice as effective or ineffective 
inevitably stems from a reasoned judgment. The. ultimate 
definition of the effective teacher does not involve discovery 
but decree ... In the final analysis a criterion is based 
not upon evidence but judgment. But let the experts speak: 

"In the development of an original criterion ... 
validity is a logical father . than a statistical 
concept." (Bechtoldt, 1947, p. 359) 

"The criterion . . . can be subjected ^to no wholly 
satisfactory empirical test of its adequacy. The 
criterion must, consequently, be logically justi- 
fiable as valid in its *own'°right . " (Brogden 6e 
Taylor, 1950, p. 160) 

". . . there is no way in which, criteria of success 
can be established on an empirical basis- The defi- 
nition of success in any activity must always be based 
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on rational considerafions (Flanagan, 1947, p. 162) 

"The ultimate criterion- of success in any duty mufiit 
always be determined on rational .grounds . There i^s -no- 
J other basis on which this choice can be made." (lliorndike, 

' 1949, p. 123) 

• k * . 

..^ "The particular statements of what constitutes a good 

, ; teacher in any particular locality are in the nature 

of policy statemetlts — emphasizing those qualities 
, which are deemed to be acceptable to the person or 

group whose thinking has dominant force. . ." (Scates, 
. 1950, p. 141) 

If this reasoning Is correct^ then our first step should, be 

directed toward scientific studies of human jujigment. Though the' 

technology- for such efforts is atlll in its infancy., new and improved 

methods show increasing promise, Hammond and'Adelman (1976), in 

"Science, Values and Human Judgment," discuss how the integration of 

fkcts and values requires the scientific study of human judgment; 

Current efforts to integrate scientific intormatipj^ Vand 
"social values in the forming- of public. poJLlcy are^^^ti^fused 
V-and defeated by ' the widespread use o£ ascientlf ici- xn^hods — 
the adversary systecj'&tid the person-oriented approaeji« The 
adversary system suffer^ from an ascientific contol^ifent to 
vic'tory rather than truth; the person-oriented a^^^ach 
suffers from an ascientific focus on* persons and^^neir i 
motives rather than on the adequacy-^o'f methods. The reason 
for the widespread use of both lies in the'faiJLure to 
^ recognize that human*' judgment './i^an be brought undepr 

scientific, rather than ad hominem ,- analysis. The argument 
advanced here is that a scientifically, socially, and 
ethically defensible means for integrating science and hum£:n 
values can be achieved » (p. 395) ^ 

c 

' The comments of Hammond* and his colleagues illustrate -A developing' 
technology and growing enthusiasm for scientific inquiry in areas 
dealing with the integration of scientific fact and human values. The 
psychological study of judgment may provide the t'echnfology needed to 
describe the policies of important educational cqjistituenciefi . Be- 



cauS^ no such technology is yet fully develppedf^v Tic^wever , the tools 
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that are available must be closely examined before they can be used 
with. confidence. > ^ . 

One framework within'which the empiriciJ.1, study, of policies could 
be conducted is a system of scales, as described by Travers (1966). 
The system would allow individuals who value different outcome^ to 
locate ppints ^f agreement and clarify differences , but* would not 
itself prescribe '-outcomes of primary importance. - Again, a study of 
the individuals to be served 'by the research would have to be under- 
taken to determine the particular policy or policies to guide the 
choice of outcome^ variables ^ 

Researchers often use "their own judgment and policies for 
choosing outcomes p although those judgments and policies are seldom 
formally described. While this procedure * may be appropriate at- times, 
it makes the comparison and cumulation of findings exceedingly difficul 
It would be helpful, it seems,-, if researche.rs would make their own 
judgments and policies -explicit and indic/ite how they compare with the . 
policies of, other groups, such as teacher educators, ele^mentax^y and 
.secondary school teachers, parents, and government officials. Research 
using outcome variables suggested by these alternative' policies might 
find a larger -appreciative audience. 

Questions in studyin)^ human judgment about effective teaching ^ Do 
teacher educators or parents have identifiable policies? Many people 
assume that they do, with minor individual differences evident between 
groups . ^ Disputes between' or among representfat ives of the various 
groups are of^en explained in terras of these. perceived differences in 
position , e.g., "Teacher educators believe that teachers should be 



expert in the subject matter areas, while teachers feel the need for 

greater emphasis on classroom management." Such statements assume 

' » fc 

that the differences within a groyp among teacher ediucatoi-s, for 

example — are smaller than the ^differences between groups be- ^ 
tween- teachers and teacher educators, for example. Unf ortunauely , there 
is little evidence to support that assumption. Similarly, although i 
many people feel they can identify the policies of different groups, 
their descriptions of those policies are based only on personal ex- 
'perience and the literature of professional groups. Th^ choice of out- 
come variables in a broad research plan should have a firmer founda- 
tion. * , 

The techniques that Hammond jand Adelman (1^76) used in their 

-> . . , ... 

psychological research on judgment were directed, at assejssing indivi- 

dual policies about type of bullets that should be used by the local 

police. Hammond determined the relatij^ weight attached to factors of 

i central importance, considered those weights to assess each of the 

bullets in question, and suggested a decision satisfactory to all 

parties. Although the elements in Hammond's study are more clear-cut 

t^han are the correspon4ing elements in the planning of research on 

teaching teachers, his work suggests a potentially productive approach. 

A naive method of determining people !s judgnu2nt*making policies 

is to ask them directly what factors they consider in making judgments.- 

Parents, for example, might be asked how they decide which teacher they^ 

prefer for their child, with the researchers probing to reveal the 

factors (or "cues") considered and their relative weights. A list of 



possible cues might be' presented to the parent to provide additional ~ 
Structure to'the procedure.. In. either case, policies of several . ' 

parents mighfc be aggregated to a' single "parint judgment policy." ' ' 

^ this approach h^s three serious flaws. First, if , the interviews^ 
are unstructured, aggregation may be very difficult; it would probably 
be difficult to extract specific cues from a lengthy nax:r^tive and 
even more difficult to compare the cues extracted across narratives. - 
In addition, people are notorious for making exaggerated claims about 
the complexity of their decision processes* 'Finally, .it is well 
knovn that people do not make accurate predictions of their own future 
behavior- In real life, a parent might choose a teacher qkite . 
different from the type indicated to the researchers. 

To avoid the drawbacks of soliciting direct descriptions, of 
judgment, psycholo'gi^s ts have adopted a more^so'phis ticated method of 
determining judgment-^making policies. They ask research su^ects to 
make judgments based on descriptions of complex si tuatioi;|^s • Statistical 
procedures are then, used to infer the relative importance of the fac- • 
tors used by the subjects in making the judgments. For example, parent^ 
might be asked to rate teachers based on a series of desciriptions . =*If 
the researcher constructed the descriptions by intentixTn^lly varying 
'a set of cues, he might use regression analysis to describe relation- 
ships between the cue§ and the judgments. Such a, sN^dy is called;^ 
polic>^-capturing study. ^- ' 

The policy-capturing approach provides at least a part;ial solu- 
tidn to the first two difficulties of the direct approach. Responses 



, are easily classified since the cues under investigation are ^deter- 
mined in advance. More importantly, a better assessment of the com- 
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plexity of judgment is obtained because the judgments, rather than 
the subjects* description of the judgments are used as data. 

The third difficulty the discrepancy between judgments made in 
research settings and "real world" judgments — may be partially over- 
come by making the research setting as similar to the "real world" as 

possible. Brunswik and his followers have advocated use of this pro- 

i' 

cedure, emphasizing the losses entailed by any other approach* In 
addition %o making the judgment situations as life-like as possible, 
they choose only combinations of 4ues which are encountered outside 
the research letting. (A competing school of research holds that the 
advantages of independently chosen cues outweigh the disadvantages 
posed. by occasional strange cue combinations.) 

Several recent studies have dealt with teacher judgments (e.g*, 
Shavelson, Cadwell, & Izu, 1277). and one has directly addressed 
judgments of teacher quality (Anderson, 1977). Anderson's study 
provides a good example of the general approach to be attempted, but 
it focuses on the general process -of judgment rather than on the factors 
contributing to particular judgments. The report centers on factors 
influencing consistency of judgment (e.g., verbal vs. numerical presenta 
tion of cues describing the teachers to be judged). In marked contrast, 
a study being considered' by the Teacher Education Research Program at 
the Institute for Research on Teaching would probe the selection of fac- 
tors used in making judgments and -compare the patterns of judgment among 
Itrtportant groups in education. 

As a look at the past failures reveals, the task of adequately 
analyzing ends Is extremely difficult. Although the introduction of 
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new technologies provides some hope for eventual success, new tech- 
nologies also introduce new difficulties. Some difficulties can be 
anticipated and should be considered. 

Problems, problems, problems > Judgment studies have only recently 
been applied to the identification of value positions. A number of 
questions still need attention before the approach can be applied 
with confidence to the choice of goals for teacher education. We 
will cite several of these problem areas (without providing solutions) 
to indicate the scope of work which must be undertaken before this 
approach can be used* 

B'irsL of all, completeness and precision of expression are desirable 
features of a system of scales, but features that are difficult to 
obtain. An ideal system would include all features that constitute 
judgments of teachers. A scale constructed only on the basis of abstracts 
from the literature can suffer from an overly narrow view. Although 
the system can always be adjusted to incorporate new material, research 
plannex-s must stop reading at some point and start planning. How will 
they know when that point is reached? 

^ The system should also provide precise characterizations of 
each of its component scales so that interpretation of the scales 
will be uniform. To understand the difficulty of this task, one need 
only consider the volumes of philosophical literature attempting to 
clarify concepts such as "liberal education" or "the inquiry approach." 
When we consider the difficulty philosophers have had in making one 
idea precise, we become aware of the pressing need for precise devices 



to simplify characterizations of the major divaensions along which 
teachers are judged • 

The successful construction of a system of scales is still im- 
portant, however, for the policy-capturing approach. Yet, one of the 
greatest difficulties in conducting a policy-capturing study is 
determining which cues to investigate. We have proposed that the 
system of scales be used as a source of cues; in that case, the 
development of a precise system must precede the choice of cues. If 
the system is inadequate, what other sources of cues sTiould be used? 

Even when the system is satisfactorily constructed, it will 
undoubtedly suggest far more cues than any single policy-capturing 
Study can investigate. If only 10 cue^ were considered (each of 
which could be either present or absent), 1,024 different possible 
descriptions could be generated. Since many scales would give a 
wider range of options, it is clear that only a small fraction of the 
system could be investigated at one time. How should the choice of 
cues for initial investigation be made, and how should additional 
scales be incorporated into subsequent investigations? 

Constr,uction of the appropricHie judgment task is also a 
difficult undertaking. It should be as realistic as possible, yet 
flexible enough to be appropriate for several different groups of 
individuals. The precise wording of thfe judgment question will un* 
doubtedly have a large influence on responses received. In preliminary 
work at the Institute for Research on Teaching, the need for precision 
and clarity was repeatedly emphasized. How should the judgment task 
be constructed, and what should that crucial question be? 

r 

Once the policy-capturing study is successfully completed, a 
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larger and more nebulous problem must be resolved. The results of 
the policy-capturing study nriast be used to select outcome variables 
in planning research on teaching teachers; the identification of 
value positions alone is insufficient for identifying outcomes. How 
will the litdc between "captured" policies and outcome variables be 
forged? 

} 

Concomitant with the preceding problem is the question of 
measurement. As the entire area of affective measurement testifies, 
important outcomes cannot always be easily measured; some people, in 
fact, suggest that only trivial outcomes can be measured precisely. 
The problem of measurement has been avoided in this discussion, but 
must be faced if practical research is to be planned. The question of' 
whether or not attention should be limited to currently measurable 
outcomes must be answered. 

Components of judgments may be determined by methods other than 
direct and indirect interview. In an ongoing study, for example*, 
lorio (Note 4) is using ethnographic techniques to identify the 

lent characteristics of the performance of a teacher highly regarded 
in the community. While addi-essing the question of what aspects of the 
teaching appear salient, Florio does not try to elicit the information 
directly from parents or other groups, as in the policy-capturing studies 
previously described. Rather, she attempts to describe the thoughts 
and actions of a teacher whose work is known to be valued by particular . 



groups. ' , 

A quite different approach to choosing goals through the study of 
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•judgments is to use favorable judgments, themselves, as goals of teach- 
er education. As is commonly done in rasearch on teacher education, 
one can design a program which would identify teachers who are high- 
ly valued by other professionals or by students. Much of the research 
reviewed by Turner (1975) uses these unanalyzed judgments as the out- 
come variable. While the policy-capturing and ethnographic approaches 
attempt to break down judgments of teachers into Xhe components 
which are desirable or undesirable, the use of judgments as^ outcomes 
makes no such attempt at analysis. Favorabl.e judgments by supervisors, 
administrators, or students are seen as desirable ends in and of 
themselves and, therefore, no analysis is necessary. In such cases, 
however, what the judges believe, or perceive, to constitute "effective 
teaching" is left unknown. 

In addition to the raethodologicai problems encountered in ob- 
tainlng accurate reports of judgments, the judgment study approach is 
subject to- two of the major criticisms of the standardized test approach. 
First, the measures used in research or program development may not 
accurately reflect the goals after the goals are determined* One 
reason why basic skills receive so much attention in educational re- 
search is that it is relatively easy to measure the extent to which tliey 
are learned. Yet, as we know, measures of achievement in the basic skill 
are soundly criticized*; one can only imagine how much stronger the 
criticism would be of measures developed to assess progress directed 
toward other goals. 

Second, and perhaps more importantly, th^^ judgment study approach, 
like the standardized test approach, seems to rest on the dubious 
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assumption that popularly-supported outcomes are valuable. As we 
have explained, valued outcomes are not necessarily valuable. Since 
tljts distinction and its ioiplicrations were previously discussed in 
depth, they will only be briefly sketched here. Basically , the 
criticism rests on the belief that most people have not given sufficient 
. attention or research to educational goals to be able to make- good 
judgments about goals. The judgments they make are probably diluted 
versions of the judgments of inflvential educators and politicians. 
(A better approach might be to examine the statements of these In- 
fluential individuals.) In any case, an appeal to the wisdom of the 
masses is not a particularly good approach to the selection of goals 
for teacher education. y 

In summary, the judgment-studies approach has the advantage 
of generating political support and popular approval. It also, 
establishes a procedure for choosing goals which can be com^^ted 
in a relatively short time period. It" can be criticized, however, on 
the grounds that it is not likely to lead to goals which can be 
practically measured and it fails- to provide valid reasons for the 
choice of goals* The final approach examined will /have quite a- dif- 
ferent set of strengths and weaknesses. ^ 

Goals Related to Social Needs 

The final *means of selecting appropriate goals of teacher 
education is to (1) determine pressing societal and community needs > 
(2) determine which needs might best be met through the educational 
system, and then (3) choose teacher education goals which will help 
meet those needs. The major advantage of thiis approach is that it 



provides an adequate justification for the goals chosen. The major 

problem is that the approach is time-consuming and difficult. In 

Teachers Make a Difference , Good, Biddle and Brophy (1975) describe 

the complicated and uniquely American (U.S,) context from which this 

problem emerged: 

Once we had little difficulty stating the goals of 
education . . . The teacher's t:ask was to train pupils in 
the time-honored Three R's, and teachers who failed in 
this mission . . were deemed by their constituents as 
failures • ^ ^ , 

Today, the problem is ho longer as simple. For one thing, . 
success in the modern world requires more than simple 
literacy. For another, the school has- grown in size and 
complexity, and it is tied to other schools as part of a 
complex system of education. As a result, today we demand 
that the school accomplish a broader range of more diffi- 
cult tasks. This would be challenge enough, but we are also 
\ less certain about what tasks should be as^^igned to ^e ^ 
school. Society is ethnically diverse and rapidly changing, so 
that we cannot be sure that the attitudes and skills we demand 
of pupils today will serve them twenty years from now . . . • 
Thus, not only i^ the catalogue of goals for the 'School broader 
and more complex, there is less^ consensus concerning these 
goals within society* 

Nor does the problem stop with the fact that we hold confusing 
goals for oiir schools. Worse yet is the fact that, we have 
few clear mechanisms for resolving conflicts among goals, 
and often we lack sufficient empirical information to know 
what to' do even if we could all agree on established goals. 
Most, other Western societies have well-established raechanisms^ 
to debate and make decisions about educational goals. Un* 
fortunately, our mechanisms for this purpose are weak. More- 
. over, as education becomes more complex, as our goals for it 

become more ambitious, educators and other citizens become more ^ 
confused about how to accomplish these' goals. Questions -cf this 
sort require research^ and too often the needed research is not 
available . .(^"^^ssis added, p. 87) 

Indeed, it seems that if practice and research in teaching and 
teacher education are to be significantly advanced, continuing and 
major efforts must be made to provide improved knowledge in this, area. 
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There are a number of strengths and weaknesses in choosing goals 

(J V 

related to social needs ^ Some of them can be readily seen by con- 
sidering how the approach might be carried out. 

Social and individual needs are, in every case, the reference 
points to which any adequate justification of an educational objective 
must be attached. Learning to add, for example, is not., in itself, 
valuable; learning to add is valuable only insofar as it helps 
meet an individual need (such as balancing a checkbook) br a social 

need (such as facilitating the transfer of goods). It is not ade- 
« 

quate to say that* a goal is valuable because some people happen to ^ 
value it; it is_ adequate to say that a goal is valuable because it 
helps satisfy a social or individual need, .Hence, the determination 
of educational goals in general, and goal^ of teacher education in 
particular, should end with a reference to individual or social needs. 

A possible first step in selecting goal'l^, then, would be to 
-identify social and individual needs. Since there is often a period 
of years between the time formal schooling ends and the time when r^eed 
alfie realized, the needs .should either be constant or "on the rise" in 
order to be relevant when the child has grown. Such needs might be 
identified by asking prominent economists, sociologists, psychologists 
and educators to apply thel.r knowledge of recent American history to ^ 
the task of predicting which social and individual needs they think 
will be most pressing in 15 years. 

There are several other possible ways of selecting goals which 
would satisfy needs. One would be to begin with an analysis of the 
present gq^ls of the school and the societal needs they are apparently 
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trying to address. Another way might be to conduct a set of historical 
inquiries that would ^eek to identify needs which have been an enduring 
part of our society. Both approaches would concentrate on our uniquely 
American- situation because we are chiefly concerned (at least in this 
paper) with education in the United States.* 

Fol^lowing the sociological functionalists (esp. Parsons, 1951), 
Schwab (Note 5) has suggested the following rubrics for organizing 
social needs'; he claims tha^ society needs to: 

1. Be reasonably coherent and conflict-resolving. 

2. a. Impart a sense of membership in some group. 

, -b. . Impart a sense of membership in a collection of 
subgroups we call American society.^ 

3. a. Produce- useful social roles for most * of its members, 
b. Have most of the useful social roles reasonably well played 

4. • Have a reasonable number of its members leading reasonably 
satisfactory lives. * . 

J 

5^ Have a reasonable quantity of expertise. 

6. Have a quantity of invention and deviation to insure 
. - Instigation of dissatisfaction with the status quo . 

These rubrics are, of course, too broad to serve as the basis for 

determining educational goals. The categories are only meant to 

suggest needs which might then lead to educational goals. 

A further factor conaes into play when attempts are made to 

determine the goals of education and teacher education through stiu!y 

of societal needs. One must consider the relative efficiency and 

■ .V ■ ... 

^ t 

*OnS might airgue that the United States is too diverse a society 
for such a study and that the scope of application ought to be re» [ 
stricted. T^ie trade-off between general applicability and congruence 
to the specific needs must be considered. 

ERIC ■ 43 
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appropriateness of schooling as compared to alternatlve^soclal Instltu- 
tlons In meeting the needs Identified. 

The task set by the social needs approach Is Indeed large. It 
would, perhaps, be unmanageable if no goals could be Identified and 

> 

used until the entire task was completed. The practical solution is 
to proceed one need or goal at a time. When a single pressing need 
is identified, associated educational goals may be established, and ^ 
alternative ways in which teacher education can be used to meet the 
goal can be investigated* Incremental rather than radical changes 
can be made in the teacher education curriculum. 

The obvious strength of this approach is that it is designed 
to provide and does provide an adequate answer to the question, 
'*Why is this educ^ational goal valuable?"* The answer will always be: 
"Because goal G leads to social or community need N being met, and 
schooling is a relatively efficacious way to meet this need," Fur- 
ther, the social needs approach would subjecC the goals of education, 
the purposes of American schools, and the expected outcom^ of school 
teaching to continuous critical examination for their realism of 
scope and for their relevance to the changing needs of American society. 

The major problems with this approach are that it is t inte-consuming 
and difficult to carry out. Unlike other approaches, this one calls 
for a continuing (in the sense of never-ending) set of inquiries. 
Even* the initial efforts would require a substantial and indeter- 
mlnate period of time for identifying various procedures and methods 
•of goal selection and specification; the identification of needs is 
difficult because an adequate definition of "need" is lacking and 
O cause virtually any acceptable definition would be hard to put into** 
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practice. It: is hard to determine the needs (as opposed to wishes or 
desires) of an individual, and even harder to determine the needs of 
a group; 

In summary, the needs approach to the identification of goals 
answers the <:rucial "why" question, but a price is'' 1) a id both in 
magnitude and in difficulty of task. 

SU^MA&Y AND CONCLUSION . _ • 

The apparent failure of research efforts to significantly improve 
teacher education programs has been blamed on an insufficient quantity 
of good research and on the failure of teacher educators to use es- 
tablished results. The solutions often $ugges zr^. niore money 'for 
research on teaching in K-12 classrooms, and improved dissemination of 
the findings to teacher educators,. * . 

A dif ferent, "analysis is suggested in this paper. vWe suggest" 
that Chree scholarly efforts , iV addition to K-IZ research, are 
necessary before research and^ development is likely to improve teacher 
education. * First, systematic development efforts raupt be undertaken 
to reexamine and translate- the K-12 research findings into articulate * 
sets of curricula for teachers « Second, research must be conducted to 
indicate how adults V* including, but not limited to, teachers can 
be taught the skills found beneficial to their professional ^activities ; 
we have called this research on professional educat^ion . Third, careful 
and systematic attent ion, must be given to the way in .which goals for 
teacher education arc determined. We call this determination of 
teacher education outcomes . The lack of previous work in these three 
research and development area^.-provides an alternative explanation for 
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the lack of researchrbased teacher education. Our suggested solution * 
to the problem is to seek support and initiate inquiry in these three 
^ domains. ... 

For each of- the? necessary research, and development activities, 
we have explained the unique contributfbn the effort would make to 

improvements in teaching practice. Further, we have provided 

* ' ' 

several different ways in which work in each area could be conducted. 
All suggested approaches ^have notable strengths and w^^knesses,' 
with no alternative considered uniformly better than the others • 
This suggests that work should be conducted on each of t^ie approaches 

, suggested. The weaknesses of one alternative will be at least partially 

\ ^ . 

overcome by the strengfehs^of the other approaches, and better balance 
can be maintained. ^ Some of the approaches will have immediate implica- 
tions for teacher education, while others will take considerably longer 
to come to fruition. Work in some areas has already begun; work in 
others has not yet even been planned. . 

Until further work is conducted on (1) the systematic de\^elopment 

- •*^> 

of K-12 research findings into curricula for teachers, (2) research on 

professional education, and (3) determination of teacher education out- 
comes, ho amount of research on K-12 teaching will significantly ad- 
vance teacher education. The research-to -practice "gap is no J.onger 

.^simply a function of inadequate findings in K-12' teaching research or ^ 

'V ' ■ ' ■ ■ 

poorly distributed publications. The most "bang for the bucks" will 
now be produced by concomitant work in the three suggested area«. 
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